The present invention relates to image coding, and in particular to variable length coding of an ordered series of quantized transform coefficients of a transform of a block of image data.
Two-dimensional variable length coding, referred to as 2D-VLC, has been widely used to code quantized transform coefficients. In traditional 2D-VLC, statistics are collected or assumed of events that include a run of consecutive zero-valued coefficients followed by a single non-zero amplitude coefficient that follows the run length. The ordering of the series of quantized transform coefficients is along a pre-selected path, e.g., a zig-zag path, in the two-dimensional path of the transform. Thus, in a typical implementation, a two-dimensional table consisting of the ending amplitude and the run-length of the preceding consecutive zero-valued coefficients is constructed and variable length codes, such as optimal Huffman codes or arithmetic codes, are assigned according to the assumed or measured statistics to form the 2D-VLC table for the subsequent encoding process. Shorter code lengths are used for the more likely-to-occur, e.g., more frequently occurring events.
2D-VLC is used in common transform coding methods, such as JPEG, MPEG1, MPEG2, ITU-T-261, etc., as follows. For motion video, an image is divided into blocks, e.g., 8 by 8 or 16 by 16 blocks. Each image is classified as interframe or intraframe. Interframe images are typically post motion compensation. The blocks of the image are transformed and the transform coefficients are quantized. The quantized transform coefficients are then coded along a specified path according to a 2D-VLC table. Interframe and intraframe images typically have different 2D-VLC tables. The DC component is typically separately encoded. Furthermore, the 2D-VLC table may be truncated so that the least frequently occurring events use an escape code followed by a fixed length code. A special “EOB” code is used to indicate the end of the block when all remaining coefficients are zero.
FIG. 1 shows how a table lookup may be used to implement a 2D-VLC scheme. Prior to the table look up, the runlength of zero amplitudes preceding any non-zero amplitude and the non-zero amplitude are determined. The table look up uses a 2D table for those likely-to-occur events encoded using variable length encoding. An escape code together with a fixed length code is used for relatively less likely-to-occur combinations.
One advantage of traditional 2D-VLC is that the position of each non-zero-valued quantized coefficient and its amplitude are coded simultaneously, which generally results in shorter code lengths than using a separate code, e.g., a VLC code for each non-zero-valued coefficient and coefficient amplitude.
Because of the widespread use of image coding, many patents have been issued on different forms of VLC. U.S. Pat. No. 4,698,672 issued Oct. 6, 1987 to Wen-hsiung Chen, one of the inventors of the present invention, for example, described one form of a two-dimensional variable length coding method.
Extensions and variations to the common 2D-VLC method are known. For example, the ITU H.263 compression standard defines one such variation sometimes called three-dimensional VLC (3D-VLC). See PCT patent publication WO 9318616 published Sep. 16, 1993 titled PICTURE DATA ENCODING METHOD and also the ITU-T H.263 standard. In 3D-VLC, each symbol (“event”) is a triplet (LAST, RUN, LEVEL) that includes: LAST, a binary flag that indicates whether or not the current non-zero amplitude-value is the last non-zero-valued coefficient in the block, RUN, the run-length of zero-value coefficients that precede the current non-zero amplitude, i.e., the number of zeroes since the last non-zero coefficient amplitude, and LEVEL, the current non-zero coefficient amplitude value. Thus, there is no need for a separate EOB codeword; whether or not the non-zero-valued coefficient is the last one is incorporated into the event.
FIG. 2 shows how a table lookup may be used to implement 3D-VLC.
One deficiency of 2D-VLC is that every non-zero-valued coefficient needs to be accompanied by a runlength code to identify its position, in the form of the number of preceding zero-valued coefficients.
In block based transform coding, there often is a region, e.g., a low-frequency region along the ordering in which non-zero-valued coefficients tend to cluster, i.e., there are often a number of consecutive non-zero-valued coefficients along the low frequency region of the pre-determined path. Each one of a number of such consecutive non-zero-valued coefficients would require the same number of codewords representing the position and amplitude.
U.S. patent application Ser. No. 10/342,537 to inventors Chen et al., filed Jan. 15, 2003 and titled AN EXTENSION OF TWO-DIMENSIONAL VARIABLE LENGTH CODING FOR IMAGE COMPRESSION describes a method called the “Extended 2D-VLC Method” herein that includes encoding repetitions of some non-zero coefficient values. One variant of the Extended 2D-VLC method provides codes for all the possible amplitude variations of consecutive coefficients that follow a set of zero-valued coefficients. This effectively reduces the runlength to 1 for all cases. The difficulty of this approach is that there are enormous numbers of patterns that can be generated from the amplitudes of consecutive coefficients. For example, with 32 quantization levels as defined in many common video coding standards, there are in the order of 32n patterns that can be generated from n consecutive coefficients. As such, in a practical implementation, only a limited number of the most likely-to-occur non-zero amplitude values, such as 1 and 2, and a limited number of lengths of consecutive non-zero-values, such as 3 or 4 consecutive values, are regrouped for pattern matching.
Furthermore, in coding, while there may be a region where there are clusters of non-zero-valued coefficients, there is also likely to be a high frequency region where any non-zero-valued coefficients are likely to be scattered.
With these observations in mind, U.S. patent application Ser. No. 10/869,229 to inventors Chen et al., filed Jun. 15, 2004 and titled A HYBRID VARIABLE LENGTH CODING METHOD FOR LOW BIT RATE VIDEO CODING, was developed to encode the position and amplitude of quantized transform coefficients separately and to take advantage of the nature of the distribution of the transform coefficients in the low frequency and high frequency regions. U.S. patent application Ser. No. 10/869,229 is incorporated herein by reference, except any material incorporated by reference in U.S. patent application Ser. No. 10/869,229 and not explicitly incorporated by reference in the present disclosure. The methods described in U.S. patent application Ser. No. 10/869,229 are each and collectively called the “Basic Hybrid VLC Method” herein.
U.S. patent application Ser. No. 10/898,654 to inventors Chen et al., filed Jul. 22, 2004 and titled AN EXTENDED HYBRID VARIABLE LENGTH CODING METHOD FOR LOW BIT RATE VIDEO CODING, was invented, and provides an alternative coding method for the high frequency region by taking advantage of the very few amplitude values in the high frequency region, especially, for example, for low bit rate and interframe applications. U.S. patent application Ser. No. 10/898,654 is incorporated herein by reference, except any material incorporated by reference in U.S. patent application Ser. No. 10/898,654 and not explicitly incorporated by reference in the present disclosure. The methods described in U.S. patent application Ser. No. 10/898,654 are each and collectively called the “Extended Hybrid VLC Method” herein.
In one embodiment of the above-mentioned Basic Hybrid VLC Method, two independent types of coding schemes are introduced to code the quantized coefficients along the path. A boundary is established along the path to define two regions, e.g., a low frequency region and a high frequency region. The boundary can be made adaptive to the video depending on a number of factors such as intraframe coding or interframe coding, standard definition television (SDTV) or high definition television (HDTV), complex scene or simple scene, high bit rate coding or low bit rate coding, and so forth. In one embodiment, the encoding of the quantized coefficients in the low-frequency region includes coding the positions of consecutive non-zero-valued coefficients and the positions of consecutive zero-valued coefficients using a run-length coding method of a first type and a run-length coding method of a second type. The encoding further includes coding the amplitude values and sign of the non-zero-valued coefficients. In the high-frequency region, in one embodiment, the encoding of coefficients in the high frequency region includes encoding the positions of runs of none or more consecutive zero-valued coefficients using a run-length coding method of a third type. The encoding further includes coding the amplitude values and sign of the non-zero-valued coefficients.
In one embodiment of the above-mentioned Extended Hybrid VLC Method, a coding method is used in the second region that takes into account that almost all non-zero-valued coefficients in the high frequency region are ±1. No amplitude coding is needed to encode runs of consecutive zeroes that end in a coefficient of amplitude 1. An exception (escape) code is included to encode those rare non-zero-valued coefficients that have values other than ±1.
Although the Basic Hybrid VLC Method and the Extended Hybrid VLC Method provide potential improvement beyond using a single 2D-VLC technique for all quantized coefficients in a block, these methods may not be optimal for various reasons, including that the dynamic nature of the quantized block coefficients may not exactly match the model assumed in a pre-determined coding technique. By the dynamic nature, we mean changed in block-to-block, and image to image in one or more of the cluster or scatter of the coefficients in a region, in whether a region has significant consecutive runs of non-zero-valued coefficients or mostly isolated non-zero-valued coefficients, in whether a region has coefficients with amplitudes dominated by a few values or even a single value, and so forth.
U.S. patent application Ser. No. 10/910,712 to inventors Chen et al., filed Aug. 3, 2004 and titled VIDEO COMPRESSION USING MULTIPLE VARIABLE LENGTH CODING PROCESSES FOR MULTIPLE CLASSES OF TRANSFORM COEFFICIENT BLOCKS, describes a method that includes classifying each block of quantized coefficients to one of a plurality of classes. Each class has a corresponding method (a coding strategy) applicable to that type of data, e.g., to the types of statistical distributions of zero and non-zero-valued coefficients, including whether the non-zero coefficients are clustered or scattered, whether the non-zero-valued coefficients, other than the DC value, are dominated with amplitide-1-coefficients, and whether or not a breakpoint can be established that defines a first region wherein the non-zero-coefficients are clustered, and a second region wherein the non-zero valued coefficients are scattered. Thus, one of a plurality of coding strategies is applied according to the nature of the coefficients. In one embodiment, the classifying compares the results of applying the corresponding strategies, and selects the class and thus the strategy according to which corresponding strategy provides the best compression. U.S. patent application Ser. No. 10/910,712 is incorporated herein by reference, except any material incorporated by reference in U.S. patent application Ser. No. 10/910,712 and not explicitly incorporated by reference in the present disclosure. The methods described in U.S. patent application Ser. No. 10/910,712 are each and collectively called the “Multi-Class VLC Method” herein.
In the Basic Hybrid VLC Method and the Extended Hybrid VLC Method, the consecutive non-zero-valued coefficients and the consecutive zero-valued coefficients in the low frequency region are coded alternatively using two independent one-dimensional variable length coding methods. It would be desirable to pair the consecutive non-zero-valued coefficients and zero-valued coefficients as a single event and apply a single variable length to the pair.
Thus there is still room for improvement in how to encode a series of quantized coefficients, in particular for a series in which there are clusters of non-zero-valued coefficients.
Thus, there is still a need in the art for variable length coding methods applicable to quantized coefficients for transform image coding. Such methods can be added to the corresponding methods in the above-described Multi-Class VLC Method.
Furthermore, one or more patents describing some existing 2D-VLC coding methods have recently been the subject of patent litigation. Thus, there is a need in the art for alternate methods that can replace commonly used 2D-VLC methods that have been the subject of such litigation.